Methods and systems for content access and distribution

ABSTRACT

A method for disseminating content over an overlay network having a plurality of routers in communication with providers and consumers of content is disclosed. A router in the overlay network receives a content descriptor corresponding to a query for content from a node, and generates a subscriber interface list based on the query for the content. The subscriber interface list is transmitted to a plurality of subsequent routers in the overlay network, wherein a content identified by the subsequent routers that correspond to the query will be routed to the router based on the subscriber interface list. The router receives first and second documents corresponding to the query for the content from the node. If the second document is the duplicate, the router deletes the second document and transmits the first document to the node or if not the router transmits both the first and second documents to the node.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/177,838, filed on Feb. 11, 2014, which is a continuation of U.S.patent application Ser. No. 12/575,016, filed on Oct. 7, 2009, now U.S.Pat. No. 8,649,382, which is a continuation of U.S. patent applicationSer. No. 11/322,828, filed on Dec. 30, 2005, now U.S. Pat. No.7,623,534, which claims priority to U.S. Provisional Patent ApplicationNo. 60/715,952, filed on Sep. 9, 2005, and further claims priority toU.S. Provisional Patent Application No. 60/715,993, filed on Sep. 9,2005, the disclosures of which are hereby incorporated by reference intheir entirety.

FIELD OF THE DISCLOSURE

This disclosure generally relates to data processing, and in particularit relates to remote data accessing using interconnected networks.

BACKGROUND

Information is increasingly being created, exchanged and stored in theeXtensible Markup Language (XML). XML is suitable for this purposebecause of its flexibility and self-describing nature. Namely, it ishuman readable, while at the same time it is convenient for machineprocessing. Examples of XML-based information include a large number andvariety of electronic newspapers, technical journals, bibliographicaldatabases, and healthcare databases. Almost every Internet web site runby large organizations with useful information content makes thisinformation available as XML now.

These information sources are typically geographically dispersed acrossa network, as are the potential consumers of this information.Communicating XML-based information presents opportunities andchallenges from a networking perspective. A network that can efficientlyforward XML data can be advantageous, as it can offload the filtering oftremendous volumes of data from consumers. Or, it can reduce the load onthe source and the network, by avoiding sources having to broadcastinformation to individual consumers. Using network layer IP multicast,while alleviating the problem somewhat, is not efficient enough becauseit results in traffic concentration, especially in an environment with alarge number of sources.

Commercially, DATAPOWER offers network appliances that accelerate XMLparsing, validation, and encryption. SARVEGA also offers an appliancefor XML subscription processing. SEMANDEX offers appliances that can beconnected into a network, and used for routing a query to relevant datasources. However, certain of these require that queries use aproprietary XML dialect for describing and querying content, and, invarious of these solutions, the queries are processed by each routeren-route.

Accordingly, there is a need for a system for content access anddistribution that addresses certain shortcomings of existingtechnologies.

BRIEF DESCRIPTION OF THE DRAWINGS

It will be appreciated that for simplicity and clarity of illustration,elements illustrated in the Figures have not necessarily been drawn toscale. For example, the dimensions of some of the elements areexaggerated relative to other elements. Embodiments incorporatingteachings of the present disclosure are shown and described with respectto the drawings presented herein, in which:

FIG. 1 is a diagram illustrating an exemplary overlay network over whichthe processes of the present disclosure may be performed;

FIG. 2 is a flowchart of an exemplary content distribution process asmay be performed over the network of FIG. 1; and

FIG. 3 is a schematic diagram of exemplary data source and routerinteractions as may be performed over the network of FIG. 1.

The use of the same reference symbols in different drawings indicatessimilar or identical items.

DETAILED DESCRIPTION OF THE DRAWINGS

The numerous innovative teachings of the present application will bedescribed with particular reference to the presently preferred exemplaryembodiments. However, it should be understood that this class ofembodiments provides only a few examples of the many advantageous usesof the innovative teachings herein. In general, statements made in thespecification of the present application do not necessarily delimit anyof the various claimed inventions. Moreover, some statements may applyto some inventive features but not to others.

Referring now to FIGS. 1-3, wherein similar components of the presentdisclosure are referenced in like manner, various embodiments of amethod and systems for content access and distribution will now bedescribed in more detail.

As more and more information is becoming available electronically, bothinformation consumers and information sources face increasingchallenges. For consumers, it is very difficult to identify sources fordesired information available over a large network, such as theInternet. For information sources, it is likewise very hard to identifyinterested consumers. The solution to both these problems is to supportcontent-based dissemination and content-based querying of information,where the network itself is responsible for identifying sources andinterested consumers for various content.

When the information need is on a topic of long-standing interest to aparticular consumer, standard information subscription systems offerconsiderable convenience in this regard. However, many of the problemsobserved with existing large-scale publishing and event-notification canbe considerably helped by having a content-oriented publish/subscribesystem, where the network itself acts as the intermediary. In such case,the network may facilitate the dissemination of the information bysetting up the appropriate distribution trees, based on the content andsubscriber interests.

When the information need is for a specific piece of information or itis on a transient or a newly emerging topic, a standard database-stylequery/response model is more appropriate than a standardpublish/subscribe model. A fundamental problem then becomes one ofidentifying the distinct information sources on the network that arerelevant to consumer content queries, and enabling seamless accessindependent of where, and across what sources, the information isstored. The framework for querying these networked information sourceshas traditionally been quite different from the publish/subscribe modelin that individual queries are posed to individual sources forprocessing. The only external support that may be available are searchengines that provide location information for the consumer. However,these does not always lend itself to providing up-to-date real-timeinformation to a query, because the location information at the searchengine may be limited and/or out of date.

Referring now to FIG. 1, an XML tree-based overlay network 100 isintroduced herein to accomplish more efficient XML content distributionvia XML routers 102, that interact with a network 110, which may be, forexample, the Internet. Publish/subscribe and query/response models areunified within the XML overlay network's infrastructure for distributingcontent made available by XML-based information sources 112 for viewingby one or more consumers 114. It additionally enables such capabilitiesas XML transformation, duplicate elimination, relevance-score basedfiltering, and access control, as described in more detail below.

This framework for the overlay network 100 lends itself to a variety ofapplications. For example, one such application described in detailherein is a “super-newspaper” application that may connect any number ofelectronic newspapers to any number of consumers 114 across the network110, each of whom provide requests or queries on news on topics ofinterest from any and all of sources 112 of the electronic newspapercontent.

As a shared infrastructure, the overlay network 100 provides a balancebetween being generic enough to support a large range of applications(as all Internet Protocol (IP) networks attempt to do) and providingcritical functionality. In the past, when the need for efficientapplication-level support within a network was desired for pervasiveapplications, various support sources were available. Examples includeL7 switches, application-aware firewalls, and Secure Socket Layer (SSL)accelerators. The introduction of application level support into anetwork is desirable when there are significant performance gains fromoffloading considerable amounts of duplicate effort at a large number ofend-systems, or when functionality rightfully belongs in a network, suchas the need for a trusted intermediary. With XML becoming a ubiquitousformat for data communication, XML-aware overlay networks according tothe present disclosure will provide support to applications thatcommunicate using this data format, for both performance reasons (e.g.,to offload filtering of data that is considered irrelevant to theend-system) and functionality reasons (e.g., access control andmaintaining privacy).

A second application for XML overlay networks are in the field ofcooperative healthcare. There are thousands of hospitals, clinics, andhealth care providers who maintain medical records of patients. Theserecords are of considerable interest to millions of patients and themedical doctors treating them, who may be interested in specificinformation (tests carried out on a specific patient in the last 2years, techniques used to detect specific diseases), but want to getthis information from all possible, relevant sources.

In these, and many other applications, there is a concerted attempt atmaking information available as XML. For electronic newspapers, the RealSimple Syndication (RSS) XML format has become the dominant format fordistributing news headlines on the Internet. For healthcare, the HealthLevel 7 (HL7) consortium is standardizing XML formats for patientrecords and other aspects of health care, and health care information isbecoming increasingly available in XML format. Content-basedsubscriptions (i.e., standing queries) and ad hoc queries against theseinformation sources can be naturally expressed in XML query languageslike XPATH.

Hereinafter, the overlay network 100 of the present disclosure will bedescribed within the context of the super-newspaper application,focusing on the functionalities required and the utility of the overlaynetwork 100 in providing these functionalities. However, it should bereadily appreciated that a variety of other environments may similarlyuse the overlay network 100 described herein.

Essentially, the super newspaper, which connects individual newspapersusing an XML-aware overlay network 100 (and example of which is depictedin FIG. 1), solves the problem of subscribers/consumers (in this case,news readers) who desire information from one or more (or even all)relevant news sources, but may not know the locations of each of thesesources. It also solves the problem of publishers (in this case, newsproviders) who desire their content to reach as many interested readersas possible, but who may not know the identity of all interestedconsumers.

For example, a sports fan who wants to continually obtain informationabout a particular player, a particular team, or a particular sport, maynot know all the newspapers that may carry news that is relevant on anygiven day. In the super newspaper application, the fan could simplysubscribe to information relating to the specific player, and all newsarticles published by any of the worldwide newspapers would be channeledto the sports fan. It is the job of the overlay network to connect allthe relevant newspaper content to the interested news readers. Ofcourse, the fan would want the super newspaper overlay networkapplication to remove duplicates, so that the same news published bymultiple newspapers (e.g., when the source is a news agency such asReuters or the Associated Press) is presented to the fan only once.

The subscription may also be very fine grained (e.g., only when theparticular team wins). Processing these types of fine grained,content-based, subscriptions requires some application layer function,which the overlay network 100 described herein provides. Without suchnetwork support, either the consumer must subscribe with each individualnewspaper (clearly a non-scalable option) or the super newspaperprovider must implement a super-site for harvesting all the news fromindividual sources and processing all the subscribers. Although GOOGLEand YAHOO are demonstrating the feasibility of the latter approach, itcreates significant traffic concentration and processing bottleneckproblems. Replication of the servers across the network would only solvepart of the problem, since the servers would still need to store all thepublished content. By offering the functionality in the overlay network100 described herein, such problems are avoided and the bar for newentrants into the content distribution is concomitantly lowered.

Prior subscription-type content distributions systems have not beenadequate in many scenarios, for example, in the case of breaking news.In this and like scenarios, an ad hoc query may be more appropriate. Forexample, when a large scale disaster strikes, it is quite difficult forfriends and relatives (who may be geographically quite distant from thedisaster) to get information from the locales close to the disaster.They may not know the names of the news publishers in that part of theworld, for example. The super newspaper application herein now enablesan ad hoc query to gather all the news from those parts of the world,and enables consumers to get the desired information without having toknow specifically the names of the individual content sources. If thetopic of the ad hoc query becomes one of continuing interest, the onetime ad hoc query can be replaced by a long term subscription.

In both the publish/subscribe and query/response scenarios, the samecontent may be added by multiple data sources. Indeed, many newspaperspublish the same report from a news agency. A substantial benefit thatthe overlay network 100 of the present disclosure can provide isduplicate elimination, where only one copy of the document is deliveredto consumers. Without this, the burden for duplicate elimination fallson the consumer itself, and also imposes unnecessary load on the networkfor delivering the redundant copies that will generally be discarded.

Related to this, another functionality that can be convenientlyimplemented by the network is relevance-score based filtering. In thepublish/subscribe scenario, this functionality filters out any contentwith relevance scores lower than the top “k” scores of documentspreviously delivered to the same clients. (A variety of well-known meansmay be used for determining a relevance score). In the query/responsecase, the network delivers only the “k” most relevant responses to thequery. By implementing this functionality in the overlay network 100,such unwanted content can be quickly identified and eliminated.

When the consumer sends a query or a subscription to particulardesignated types of content, the super newspaper is an application thatseeks to aggregate, or otherwise process, relevant news items. It isnatural to request that the returned results be transformed to attainsome homogeneity, for example, that all the images be in-line within theXML content for ease of display. Further, different contentrepresentation may be appropriate for different devices utilized by thevarious consumers. Such content transformations are naturally expressedusing existing XML query/transformation languages such as XSLT andXQUERY. Supporting this functionality in the overlay network 100relieves individual applications from the need to implement it anew.

Another function that the overlay network 100 can offer is fine-grainedaccess control. XML allows access control to be applied at a finegranularity, namely at the individual XML element level. While thisfunctionality may not be relevant to certain super newspaperapplications, it is very important in cooperative healthcareapplications. For example, an XML document containing patientinformation may be delivered in full when requested by a medical doctor,and in an abridged form, without the sensitive medical history, whenqueried by hospital administration. Note that abridging a document forthe purpose of access control is a particular case of contenttransformation.

Content sources routinely offer access control functionality and filtercontent sent to consumers according to the consumer's user credentials.However, with a large and growing number of content sources on theInternet, and with increasingly complex interactions, managing accesscontrol rules across this variety of (sometimes administrativelyindependent) content sources grows ever more problematic. The overlaynetwork 100, acting as a trusted intermediary, is a logical point tosupport access control in a manner that is systematic, coherent, andhard-to-bypass.

The key to the protocol framework is the need to support both apublish/subscribe as well as a query/response model for informationdissemination in a scalable manner When every overlay router in anetwork has to perform matching and filtering of XML content frompublishers, it becomes an impediment to making the framework scalable.In prior packet networks, the fundamental building block for fast,scalable forwarding was to execute a lookup at each node in as simple amanner as possible, such as a longest prefix match or the lookup of ahash identifier (ID). The approach herein also recognizes that multicastoffers a sound basis for minimizing the number of duplicate copies of apiece of data that traverse a link. However, a multicast framework suchas IP multicast is limited in scalability because the global limit onthe number of multicast groups supported. One solution for use herein isto exploit overlay multicast, with the flexibility of having a group foreach distinct content description, so that e very fine graineddistribution trees can be provided on an as-needed basis.

A fundamental question that arises in any publish/subscribe orquery/response model is the basis for matching publications/data withsubscriptions/queries. To address this, the overlay network 100 usescontent descriptors (CDs), described in greater detail below. Eachpublication, or piece of content, is associated with a set of contentdescriptors that contain keywords or the like corresponding to thecontent (e.g., “./international/politics/mideast. for articles aboutinternational and mideast politics, or ./USA/business/energy/oil.” forarticles on the American oil and energy businesses). Analogously,subscriptions/queries may also be content descriptors, possibly withwildcards (e.g., “./international/*/mideast.” queries could matchcontent descriptors “mideast business,” “mideast politics,” and thelike). Content descriptors are hierarchical in nature (as the examplesabove illustrate), akin in some respects to dimensions in datawarehouses, topic hierarchies in ontologies, and XML data paths.

In the processes described herein, publishers 112 and consumers 114advertise sets of CDs to the overlay network 100. For thepublish/subscribe model, the publisher's data is forwarded to all theconsumers/subscribers interested in the CD. For the query/responsemodel, CDs in the form of queries are routed to the appropriate datasources.

Wildcard-free CDs form the basis for a content-based multicast groupthat enjoins publishers 112 of the CD and the consumers 114 for that CD.For every such CD, the overlay network 100 effectively constructs twocore-based multicast trees with one core leading to all of thepublishers 112, and one that leads to all of the consumers 114. Theoverlay network creates a coordinator for each such CD, selecteddynamically, based on the first arrival of the CD from a publisher 112.The coordinator acts like a core in the core-based IP multicastframework.

As soon as a CD (from the publisher 112 or from a consumer 114) is sentto the overlay network 100, it is mapped into a set of small hashidentifiers (e.g., a single identifier for a wildcard-free CD) at afirst overlay router 102 of the network 100. This enables efficientmapping to the corresponding multicast trees for forwarding CDs ofqueries or content. Subsequent routers beyond the first (coordinator)router only forward CDs based on the hash identifier (ID) on theappropriate multicast tree.

In various embodiments, the functionality of the overlay network 100 andits routers 102 may be described generally by the exemplary flowchart ofFIG. 2, in which one exemplary process 200 for disseminating contentaccording to the present disclosure is displayed. The process 200commences when a CD is received at a first router 102 in the overlaynetwork 100 (step 202). The first router then determines whether the CDis a duplicate entry (step 204). If the CD is a duplicate, the process200 continues directly to step 208 below. Otherwise, the process 200continues to step 206 wherein the router designates itself as thecoordinating router for the CD and generates a tuple corresponding tothe CD. The tuple is then forwarded to other routers 102 in the overlaynetwork 100 (step 208), thereby forming a tree, or path, for contentrelated to the CD to be identified and transmitted. The process 200 thenends.

Particular functions of the overlay network 100 may be explored withrespect to FIG. 3 wherein, various overlay XML routers (R), such asrouters 102, and data nodes (D), which may include publishers 112 andconsumers 114, are diagrammatically depicted, wherein each XML routerknows the overlay network's topology.

One characteristic of the protocol used by the overlay network 100 isthe creation of a coordinator for each CD that anchors furtherdistribution tree constructions. The overlay XML router (R) next to thefirst data node (D) that transmits a particular CD elects itself as thecoordinator for that CD and floods that information throughout theoverlay network 100. In order to avoid data congestion, when there areparallel attempts by multiple coordinators to flood a CD, thecoordinator with, for example, the smallest ID may be allowed toproceed. Other methods of electing a prevailing coordinator may likewisebe used.

So, for example, if the data node labeled D3 in FIG. 3 publishes a dataitem with CD=<ABC>, the nearest overlay XML router, labeled R2, electsitself as the coordinator for that CD. The coordinator constructs a hashvalue based on the CD, and then floods the tuple (hash ID, <ABC>,coordinator-ID) to all the overlay routers in the network. Thecoordinator ID lists the ID of the coordinating router for the CD. Thestate stored in all the overlay XML routers 102 of the overlay network100 includes a corresponding tuple (hash ID, <ABC>, publisher interfacelist), where the last item contains the list of interfaces towards thepublisher 112 for this CD. For the non-coordinator routers of theoverlay network 100, the publisher interface list may be a list ofinterface to the coordinator router. For the coordinator router, thepublisher interface list may include a list of interfaces to the datanode D3 that published the CD.

If subsequently, another data node D4 publishes the same CD=<ABC>, theadjacent overlay router R4 sends a publisher-join command to add theappropriate interfaces on its publisher interface list. In the presentexample, this means sending the command through the overlay network'stopology towards the coordinator router, R2. This goes to R3 to R2, andin various embodiments, along the shortest path of overlay hops. Whenthe router R3 receives the publisher-join command, it adds its linktowards R4 to its publisher interface list for this CD. Similarly,overlay XML router R2 adds the link towards R3 in its publisherinterface list for this CD. Thus, we have a tree of publishers formed.FIG. 2 reflects the routing state at this stage, with the firstbracketed list at each router representing the publisher interface list.

Subsequent CDs that are announced by other publishers may follow theshortest path of overlay hops towards the coordinator router, but do notneed to progress all the way to coordinator router once they hit an“on-tree node.” For example, if D6 published a document with <ABC>, itspublisher interface list would stop at R3 because the latter is anon-tree node. A router determines that it is an on-tree node if itspublishers interface list contains a similar entry for the CD.

In another example, if a consumer on node D1 sends a query for <ABC>,the first overlay XML router, R1, adjacent to D1, processes the queriedXML path to derive a hash function (or potentially a set of hashes ifthe submitted query contains multiple paths) to match what was computedearlier for that CD. With the hash, the overlay XML router R1 now knowsthe relevant tree, and forwards the message containing the hash andquery according to the interfaces in the publisher interface list. Notethat this list is guaranteed to include the interface towards thecoordinator router. In the present example, the query is sent towardsthe coordinator router R2. R2 will in turn send the query on itspublisher interface list, which includes D3 and R3. R3 will forward thequery to R4, and then to D4. Thus, the query will be delivered to allrelevant publishers.

One characteristic the overlay network 100 is that only the firstoverlay XML router 102 in the overlay network 100 that receives a queryhas to process the query. The remaining routers do not have to processthe query, and just use the hash ID to determine where to send thequery, similar in result to traditional multicasts, along all theinterfaces that are marked with senders for that hash ID. Use of thehash ID therefore avoids the processing of queries at all the routers102. Instead, this allows each subsequent router to perform a simplehash lookup in place of full processing.

Each data source receiving a CD from the overlay network 100 may, ofcourse, process a received query in its entirety. The source may havemultiple documents that match the query. The data source (e.g.,publisher 112) may not have exposed the entire internal structure of thedocument in the original CD announcement. Thus, it is useful for thedata source to process the entire query. Access control may beimplemented at the data source, based on which consumers 114 may in factobtain the query response.

When a consumer 114 sends a subscription corresponding to a CD, or agroup of CDs, to an adjacent overlay router, a subscriber-join commandmay be multicast to publishers 112 over an appropriate publisher tree inthe overlay network 100. In the process, the subscriber-join also setsup a second logical tree (subscriber tree), along which published datamatching the subscription will flow. Thus, the routing state entrydescribed in the previous subsection is augmented with a subscriberinterface list. Each overlay XML router 102 receiving a subscriber-joinadds the interface from which the subscriber join was received to thesubscriber interface list for the received CD hash. As withpublisher-joins, the propagation of subscriber-join messages stops atoverlay routers 102 that are already on the subscriber tree (e.g., onewhich has a subscriber interface list with the appropriate list). When anew publisher 112 arrives on an interface of a router 102 that alreadyhas an existing subscription for a CD (e.g., the overlay router is anon-tree node for that subscriber), then the subscriber join is sent tothe overlay XML router adjacent to that publisher 112, thus extendingthe subscriber tree up to the appropriate publisher 112.

In the example of FIG. 3, assume that D5 wants to subscribe to updatesto the content described by the CD=<ABC>. This subscriber-join commandis sent to R4, the nearest overlay router. The state for thesubscription tree is created at R4, which forwards along the publisherinterface list to router R3. R3 creates the state for this subscriptiontree and forwards it to R2. A subsequent subscription from node D6 forthe same CD (an identical subscription to the previous one) will stop atR3 because it is an on-tree node for the subscription tree. R3 adds D6to its subscriber interface list. However, there is no need to propagatethe subscription further, since all publication of content correspondingto CD=<ABC> will be forwarded to R3, and thereafter by R3 to D6.

The routing state at this stage is shown in FIG. 3, with the secondbracketed lists representing the subscriber interface lists. There areseveral cases to be considered, especially those involving subscriptionswith wild-cards.

For example, if the subscription is subsumed by an existingsubscription, the new subscription is sent towards the coordinatorrouter of the more general subscription and joins the tree of thesubsuming subscription. If, however, this subscription partiallyoverlaps with another subscription, the new subscription may instead besplit into two parts: one that is totally subsumed by anothersubscription and the remaining unique part. The router then sends thesubsumed part towards the pre-existing coordinator of the subsumingsubscription and joins the corresponding tree. The router closes to thesubscriber may also become the coordinator of the unique part. If,alternatively, the subscription does not overlap with any existingsubscriptions, or the overlap is too small to yield significantperformance improvements, the first router acts as a new coordinatorrouter for the unique CD.

It is highly desirable to suppress duplicate response to queries ofmultiple databases, as well as the same content being sent by multiplepublishers. Duplicate elimination for responses to queries may beoptional, and may or may not be specified in the query or response(e.g., via a flag) to direct the overlay network to eliminateduplicates. For performing duplicate elimination, it is assumed thatthere is a reasonably accepted notion of which fields to examine todetermine that the document is a duplicate. For example, the date, titleand author or source of the document may be used when determiningwhether it is a duplicate of an already seen document. The network mayuse a Message Digest 5 (MD5) hash of the content to match forduplicates. Alternatively, the overlay XML network router may build anXML tree of a newly received document. If its XML tree is subsumed by apreviously received document's XML tree, as per any of a variety ofuseful similarity tests, then the newly received CD for such content isnot forwarded. Otherwise, the newly received CD may be integrated in theoverall XML tree structure. When taking advantage of the XML structureof the content, the additional task that may have to be performed by theoverlay router would be to construct a hash for the content at theleaves of the XML tree structure. The closest router 102 in the overlaynetwork 100 to a publisher 112 responding to the query would be the oneto generate the hash.

The first response is delivered to the consumer or subscriber 112generating the query. The state associated with duplicate eliminationhas to be retained at the overlay network routers until all responseshave been received. Of course, the amount of time needed to receive allthe responses is generally not known. Therefore, the network may use atimer or the like to determine when an overlay network router stopslooking for duplicates and discards the state built up. Every router inthe overlay path from the first overlay router adjacent to thepublisher/content would have to maintain the timer, and each responsereceived (whether duplicate or not) would reset the timer at the overlayrouter. While the timer is ON, duplicate elimination would occur, andthe overlay router would maintain state associated with each distinctcopy. When the timer expires, then the state associated with theresponse (or published content) would be discarded.

Thus far, only responses that are exact matches to a query have beenexplored. When a document contains multiple items, and it matches onlyapproximately with the content, users may need the network to presentonly the top ranked responses (up to some number “k”). Various methodsfor ranking the top k responses may be readily adapted and used herein.

Regarding responding to queries in a ranked manner, the first routerthat receives an initial response may start a priority queue structurefor the response based on the pre-specified ranking system, and forwardsthe response towards the appropriate consumer(s) or subscriber(s). Anysubsequent response that arrives related to the query (or subscription)updates the priority queue and only messages that have a higher rank aresent out. All the intermediate overlay routers continue to forwardresponses that have a higher ranking. It is up to the router that isadjacent to the appropriate consumer/subscriber to queue all theresponses received, and finally release only the top k responsesreceived. Only this overlay router would need to maintain a timer, whichensures that all the responses have been received.

One observation to note is that, at any hop in the overlay network 100,the messages going out will not result in saving significant bandwidthon the links, if the rank of the received messages is monotonicallyincreasing. However, this should occur infrequently. In any case, sincethe overlay router adjacent to the subscriber buffers all the receivedresponses until it has received all responses (based on the timer atthis node), the subscriber only receives the top k responses, and doesnot have to perform additional filtering.

In various embodiments, the overlay network 100 may include variousknown and developing facets, including recent advances in XMLprocessing, such as XML indexing, XML filtering andpublication/subscription support, XML transformation and XMLtranscoding. In addition, languages having similar structure orfunctionality to XML, or later developed versions of XML, may be usedwith suitable adaptation in the overlay network 100 of the presentdisclosure.

Although the best methodologies have been particularly described in theforegoing disclosure, it is to be understood that such descriptions havebeen provided for purposes of illustration only, and that othervariations both in form and in detail can be made thereupon by thoseskilled in the art without departing from the spirit and scope thereof,which is defined first and foremost by the appended claims.

What is claimed is:
 1. A system, comprising: a memory that storesinstructions; a processor that executes the instructions to performoperations, the operations comprising: determining if multiplecoordinator routers are attempting to flood a first content descriptorin an overlay network; selecting, if the multiple coordinator routersare attempting to flood the first content descriptor, a first router asa coordinator router for the first content descriptor based on the firstrouter having a smallest hash identifier when compared to other routersof the multiple coordinator routers; generating, by utilizing the firstrouter, a tuple comprising the first content descriptor whichcorresponds to first content; transmitting, by utilizing the firstrouter, the tuple to a second router so as to form a path for the firstcontent to be transmitted; and transforming, by utilizing the overlaynetwork and based on a credential of a user requesting the firstcontent, the first content by filtering a portion of the first contentfrom the first content.
 2. The system of claim 1, wherein the operationsfurther comprise generating a publisher interface list.
 3. The system ofclaim 1, wherein the operations further comprise receiving, at thesecond router, a query that corresponds to the first content descriptor.4. The system of claim 3, wherein the operations further compriserouting the query to the first router.
 5. The system of claim 1, whereinthe operations further comprise receiving, at the second router, asecond content descriptor corresponding to second content.
 6. The systemof claim 1, wherein the operations further comprise identifying, byutilizing the first router, the first router as the coordinator router.7. The system of claim 1, wherein the operations further compriserouting the first content descriptor to other routers in the overlaynetwork according to the tuple.
 8. The system of claim 1, wherein theoperations further comprise determining whether the first contentdescriptor is a duplicate entry.
 9. The system of claim 8, wherein theoperations further comprise transmitting the tuple to the second routerafter determining that the first content descriptor is the duplicateentry.
 10. The system of claim 1, wherein the operations furthercomprise receiving the first content descriptor.
 11. The system of claim1, wherein the operations further comprise filtering an additionalportion of the first content from the first content based on a relevancescore of the additional portion of the first content.
 12. The system ofclaim 1, wherein the operations further comprise determining a relevancescore for the first content.
 13. A method, comprising: determining, byutilizing instructions from a memory that are executed by a processor,if multiple coordinator routers are attempting to flood a first contentdescriptor in an overlay network; selecting, if the multiple coordinatorrouters are attempting to flood the first content descriptor, a firstrouter as a coordinator router for the first content descriptor based onthe first router having a smallest hash identifier when compared toother routers of the multiple coordinator routers; creating, byutilizing the first router, a tuple comprising the first contentdescriptor which corresponds to first content; forwarding, by utilizingthe first router, the tuple to a second router so as to form a path forthe first content to be transmitted; and transforming, by utilizing theoverlay network and based on a credential of a user requesting the firstcontent, the first content by filtering a portion of the first contentfrom the first content.
 14. The method of claim 13, further comprisingdetermining a relevance score for the first content.
 15. The method ofclaim 13, further comprising distributing the first content over theoverlay network.
 16. The method of claim 13, further comprisinggenerating, at the first router, a hash of the first content descriptor.17. The method of claim 13, further comprising determining whether thefirst content descriptor is a duplicate entry.
 18. The method of claim13, further comprising routing the first content descriptor toadditional routers in the overlay network based on the tuple.
 19. Themethod of claim 13, further comprising receiving, at the second router,a query that corresponds to the first content descriptor.
 20. A system,comprising: a memory that stores instructions; a hardware processor thatexecutes the instructions to perform operations, the operationscomprising: determining if multiple coordinator routers are attemptingto flood a first content descriptor in an overlay network; selecting, ifthe multiple coordinator routers are attempting to flood the firstcontent descriptor, a first router as a coordinator router for the firstcontent descriptor based on the first router having a smallest hashidentifier when compared to other routers of the multiple coordinatorrouters; transmitting, by utilizing the first router, a tuple to asecond router so as to form a path for first content to be transmitted,wherein the tuple comprises the first content descriptor whichcorresponds to the first content; and transforming, by utilizing theoverlay network and based on a credential of a user requesting the firstcontent, the first content by filtering a portion of the first contentfrom the first content.